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DETAILED ACTION 

1. The Art Unit location of your application in the USPTO has changed. To aid 
in correlating any papers for this application, all further correspondence regarding 
this application should be directed to Art Unit 2483. 

2. This Office action for U.S. Patent Application No. 10/802,428 is responsive to 
communications filed 28 July 2010, in reply to the Non-Final Rejection of 28 April 
2010. Claims 1-3, 5, 7-15, 17, and 19-25 are pending. 

3. In the previous Office action. Claims 1-3, 5, and 7-12 were rejected under 35 
U.S.C. § 101 as non- statutory. Claims 1-3, 5, 7, 10-15, 17, 19, and 22-25 were 
rejected under 35 U.S.C. § 103(a) as obvious over "Motion-based Segmentation 
Using a Thresholded Merging Strategy on Watershed Segments" ("de_Smet") in 
view of "K-Harmonic Means-A Data Clustering Algorithm" ( "Zhang "). Claims 8, 9, 
20, and 21 were rejected under 35 U.S.C. § 103(a) as obvious over de Smet in view of 
Zhang and in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" ( "Hermann" ). Claim 27 was rejected 
under 35 U.S.C. § 103(a) as obvious over de Smet in view of Zhang and in view of 
U.S. Patent No. 6,084,912 A ("Reitmeier"). 

Response to Amendment 

4. Applicant's amendment to Claim 1 has been fully considered. The rejection of 
Claim 1 under 35 U.S.C. § 101 is withdrawn. 
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Response to Arguments 
5. Applicant's arguments filed with respect to the prior art rejections have been 
fully considered but they are not persuasive. Applicant argues that the cited de 
Smet and Zhang references do not disclose the claimed limitations of "estimating 
motion paths" and "calculating values representing errors". The examiner 
disagrees. The de Smet reference is directed to a "motion-based segmentation" (§ 1) 
in which an "initial motion field" is first generated (§2.1). In generating the initial 
motion field, a series of estimated motion vectors are generated. Id at § 2.1. This 
"motion field" comprising motion vectors is a set of the claimed "motion paths", and 
the "initial" motion field is the one generated in the initialization step. In 
segmentation, segments of an image are merged "based on the similarity of [the] 
motion vectors". Id at § 2.3. In this application of the K-means clustering method, 
a difference between motion vectors would be a measure of similarity, with a 
smaller difference meaning a greater degree of similarity. In particular, de Smet 
checks to see if the "distance" between neighboring motion vectors is "below a 
certain threshold". Id. To use the language of the claim itself, a difference or 
distance between motion vectors is the claimed "error". 

Since the K-Harmonic Means algorithm is useful in any application of the K- 
means algorithm, including "data compression and vector quantization" (Zhang § 1), 
if the Zhang algorithm discloses comparing differences, distances, or errors between 
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two vectors or data sets that represent vectors in this application as a condition of 
merging segments, the Zhang algorithm discloses the claimed step of calculating 
errors. It is noted that the current amendment to the claims replaces the phrase 
"distances" with "values representing errors". This does not appear to be a 
substantial amendment changing the scope of the claim, but rather a clarifying 
amendment with respect to the particular application of "estimating motion trials in 
video image sequences". As such, the previous interpretation of d(i,l) in Zhang as 
the previous "distance" incorporates the claimed "error". In the particular 
application of Zhang to segmenting an image based on motion, as in de Smet , the 
"center points" are believed to be the endpoints of the motion vectors in the field 
described in de Smet . Therefore, the combination of the de Smet and Zhang 
references, as a base clustering function described as useful for segmenting images 
based on motion, and the improved clustering function specifically designed as an 
improvement of the base function, is still considered obvious. The claim rejections 
are maintained and this action is made final. 

Claim Rejections - 35 USC § 103 
6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
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matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

7. Claims 1-3, 5, 7, 10-15, 17, 19, and 22-25 are rejected under 35 U.S.C. 
103(a) as being unpatentable over "Motion-based Segmentation Using a 
Thresholded Merging Strategy on Watershed Segments" ( "de Smet" ) in view of "K- 
Harmonic Means-A Data Clustering Algorithm" ("Zhang")- De Smet discloses using 
an iterative segment-merging technique to determine information for an image 
(abstract). 

Regarding claims 1, 13, and 25, in de Smet, an initial motion field is first 
determined with a block- matching technique on 4 x 4 blocks. De Smet , § 2.1. These 
initial block motion vectors are used for the initial segmentation. Id. at § 2.3. 
Then, the step of performing the block-based motion estimation is the claimed step 
of "providing data points", and the motion vectors themselves in the initial motion 
field are the claimed "motion paths". Next, the segments are iteratively merged 
according to similar or shared motion, according to the K-means clustering 
algorithm. Id This is the claimed step of "cluster[ing] the data points". When this 
process is finished, the result is a series of large segments corresponding to distinct 
moving regions of an image, each with an associated motion vector. Id. This is the 
claimed step of "providing motion estimation". However, the present invention 
specifies performing regression clustering according to a K-Harmonic Means 
function, which is not the same as the K-means function of de Smet . 
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Zhang discloses the K-Harmonic Means data clustering algorithm. 
Regarding claim 1, 3, and 25, Zhang teaches selecting K centers m(l) from N data 
points x(i) (pg. 1), initializing center points (pg. 2) and performing an initial 
iteration (pg. 5), and calculating distance d(i,l) between data point x(i) and center 
point m(l) (pg. 4), calculating membership probability p(i,x) based on the distance 
via parameters q(i,k) and q(i) (pg. 5), and stopping when the recursively-calculated 
performance value stabilizes, that is, when its change with each iteration becomes 
small (pg. 5). Then, the K calculations of centers m(l) are the claimed "regression 
functions" for performing regression clustering according to the K-harmonic means 
function. The calculation of each iteration of the recursive function is the claimed 
recalculation based on the membership probability, as m(k) is dependent on 
probability function p(i,k). The stabilization is the claimed "stopping criterion". In 
the particular application of "data compression and vector quantization", these 
center points are the claimed "motion paths" or motion vectors of de Smet , and 
distance d(i,l) is the claimed "error". 

De Smet discloses the claimed invention, except for using K-Harmonic Means 
function to perform regression clustering. Zhang teaches that it was known to 
perform data clustering with the K-Harmonic means function. Therefore, it would 
have been obvious to one having ordinary skill in the art at the time the invention 
was made to perform clustering based on a K-Harmonic means function, as taught 
by Zhang , rather than a linear function such as the K-means function of de Smet. 
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since Zhang states in the abstract that K-harmonic clustering is less sensitive to 
detrimental effects from sub-optimal initialization than conventional clustering 
techniques. Additionally, the K-harmonic means function was specifically designed 
as an improved version of the K-means function, which "significantly improves the 
quality of clustering results" compared with K-means (abstract). Then, the de Smet 
algorithm contains a basic method which the present invention is an improvement 
thereof. The prior art Zhang describes a known technique, the K-Harmonic means 
clustering algorithm, applicable to the base segmentation method of de Smet . by 
substituting it for the K-means algorithm. Then, one having ordinary skill in the 
art would have recognized that applying the K-harmonic means algorithm of Zhang 
to de Smet would have yielded the predictable result of "significantly [improved] 
quality of clustering results" and resulted in an improved system. Therefore, it is 
respectfully submitted that the use of K-harmonic clustering in de Smet is 
considered obvious, since it has been held that applying a known technique to a 
known method ready for improvement to yield predictable results involves only 
routine skill in the art. Dann v. Johnston . 425 U.S. 219, 230, 189 U.S.P.Q. 257, 261 
(1976); In re NiUsen . 851 F.2d 1401, 1403, 7 U.S.P.Q.2d 1500, 1502 (Fed. Cir. 1988). 



Regarding Claims 2, 3, 14, and 15, as previously mentioned, de Smet 
produces a motion vector for each segment in an image. De Smet, § 2.3. As a result. 
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the most important moving areas are determined. Id. at § 3. This is the claimed 
production of motion vectors and at least one motion path. 

Regarding Claims 5 and 17, in Zhang , a clustering in which initialization is 
randomized is described (pg. 11). Since Applicant admits in page 10 of the 19 
January 2010 Appeal Brief that this initialization is an initialization of the 
algorithm as a whole, it inherently contains an initialization of the regression 
function. 

Regarding Claims 7 and 19, insensitivity to initialization is an inherent 
result of the K-Harmonic Means algorithm. Zhang , abstract, i 

Regarding Claims 10 and 12, in de Smet, pixels are set as (x, y, t) triples, with 
X and y as spatial coordinates and t as a time coordinate. De Smet, § 2.2. 

Regarding Claims 11 and 23, de Smet illustrates motion fields. De Smet, figs. 
3-6. Although these motion fields are not shown as overlaid on the images, the 
examiner takes Official Notice that it was well-known in the art at the time of the 
invention to display a motion field superimposed on an image to provide a visual 
representation of motion vectors. 

Regarding Claims 12 and 24, de Smet illustrates highlighted motion 
segments overlaid on an image. De Smet, figs. 11, 12. 



1 The rejection of Claims 7 and 19 was inadvertently omitted in the previous Office action. Since 
Applicant did not specifically allege claims 7 and 19 to be allowable on their own merits in the 28 
July 2010 arguments or the 19 January 2010 appeal brief, nor allege that the omission of this 
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8. Claims 8, 9, 20, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over de Smet in view of Zhang as applied to claims 1 and 13 above, 
and further in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" (Hermann). Claims 8, 9, 20, and 21 
disclose using color information to segment techniques, but de Smet only discloses 
"standard watershed techniques" to perform initial segmentation without providing 
details. De Smet, § 2.2. 

Hermann discloses a method for image segmentation to extract objects from a 
moving image. Regarding Claim 8, 9, 20, and 21, after an initial block- matching 
motion estimation, similar to that in de Smet, images are segmented according to 
specific color information, followed by shape analysis, and lastly motion analysis to 
merge regions to determine objects. Herrmann, § II. This color, shape, and motion 
information form the claimed "predetermined criteria". In color analysis, a region is 
determined as homogeneous if the pixel difference in the region is below a 
threshold. Homogenous, connected areas are determined as "quasi-flat zones". 
These quasi-flat zones are further processed and become the basis for further 
segmentation. Id. at § II.B. Then, the color analysis is the claimed step of 
"portioning data according to color". 

De Smet, in combination with Zhang, disclose a majority of the features of 
claims 8, 9, 20, and 21 of the claimed invention, as discussed above, except for color 



rejection is a concession of allowability in the 28 July 2010 arguments, the examiner assumes that 
Applicant recognizes that this omission was unintentional. 
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segmentation. Herrmann teaches that it was known to segment a moving image 
according to color. Therefore, it would have been obvious to one having ordinary 
skill in the art at the time the invention was made to determine image 
segmentation by color as taught by Herrmann, since Herrmann teaches in p. 205, 1 
3, that color analysis produces the most accurate type of segmentation. 

Conclusion 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the mailing date 
of this final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to David N. Werner whose telephone number is 
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(571)272-9662. The examiner can normally be reached on Monday- Saturday from 
10:30 to 5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Joseph G. Ustaris can be reached on (571) 272-7383. The fax 
phone number for the organization where this application or proceeding is assigned 
is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571- 
272-1000. 

/D. N. W./ 

Examiner, Art Unit 2483 

/Joseph G Ustaris/ 

Supervisory Patent Examiner, Art Unit 2483 



